ISyE 8843 A , Brani Vidakovic Handout 21 Wavelet Based Nonparametric Bayes Methods

Wavelets are the building blocks of wavelet transformations the same way that the functions einx are the building blocks of the ordinary Fourier transformation. But in contrast to sines and cosines, wavelets can be (or almost can be) supported on an arbitrarily small closed interval. This makes wavelets a very powerful tool in dealing with phenomena which change rapidly in time. Statistical wavelet modeling and computational research has, in recent years, become a burgeoning area in both theoretical and applied statistics, and is beginning to impact developments in statistical methodology and in various applied scientific fields. Wavelet ideas are developing in statistics in areas such as regression, density and function estimation, factor analysis, modeling and forecasting in time series analysis, spatial statistics, with ranges of application areas in science and engineering. The emerging interests in Bayesian statistical modeling and wavelets is generating exciting new directions for the interface of two research areas, with significant potential for future impact on applied work. Many nonparametric procedures are in fact infinitely parametric. An example is the orthogonal series regression or density estimator. In order to estimate such functions, shrinkage, tapering or truncation of parameter estimators from an infinite class is necessary (Chencov’s orthogonal series density estimators, Stein-type estimation, and so on.). Wavelet shrinkage is a simple and yet powerful tool in nonparametric statistical modeling. It can be described as a three step procedure. Data are transformed into a set of wavelet coefficients, a shrinkage of the coefficients is performed, and then the shrunken wavelet coefficients are transformed back in the domain of the original data. Wavelet domain is good modeling environment; several supporting arguments are listed below. Discrete wavelet transformations tend to “disbalance” the data. Even though the transformations preserve the `2 norm of the data, the “energy” (an engineering term for the `2 norm) of the transformed data concentrates in only a few wavelet coefficients. That narrows the class of plausible models and facilitates the thresholding. Mallat (1989) gives an interesting discussion on modeling from the signal-processing point of view. The disbalancing property also yields a variety of criteria for the best basis selection. Standard references are Coifman and Wickerhauser (1992), Donoho (1994) and Wickerhauser (1994). Wavelets, as building blocks in modeling, are localized well in both time and scale (frequency). Signals with rapid local changes (signals with discontinuities, cusps, sharp spikes, etc.) can be well represented with only a few wavelet coefficients. This is, in general, not true for other standard orthonormal bases which may need many “compensating” coefficients to describe discontinuity artifacts and to suppress Gibbs’ effects. Heisenberg’s principle states that in modeling time-frequency phenomena one can not be precise in the time domain and in the frequency domain simultaneously. Wavelets automatically trade-off the timefrequency precision by their innate nature. The parsimony of wavelet transformations can be attributed to the ability of wavelets to handle limitations in the Heisenberg principle in a data-dependent manner. Also, there is theoretical and empirical evidence that wavelet transformations tend to simplify the dependence structure in the original signal. It is even possible, for any given stationary dependence in input signal, to construct a biorthogonal wavelet basis such that the corresponding wavelet coefficients become uncorrelated (a wavelet counterpart of Karhunen-Loève transformation). For a discussion and examples see Walter (1994). These arguments identify wavelet bases as suitable tools for effective statistical modeling. More favorable arguments can be given: computational speed of the wavelet transformation, simple descriptors of self-similarity and so on.